Across scientific disciplines, most domains are affected by the uncertainty associated with model conceptualization. Technically referred to as model selection uncertainty, this uncertainty is associated with correctly selecting a set of domain-specific interpretations, processes, and mathematical systems in order to accurately model behavior or classify events of interest (e.g. protein structures, weather systems, image recognition tasks, etc.). This type of uncertainty is the greatest source of error and risk associated with modeling and forecasting.
One of the most powerful ways to address model selection uncertainty is through the aggregate prediction of a model ensemble. These ensembles are composed of individual models (here models can be an algorithm, a mathematical model, and expert opinion, etc.) where each model uniquely explores a portion of hypothesis space by uniquely defining a set of processes, systems, and relationships that can describe the event of interest. The aggregate obtained from a model ensemble—through techniques such as bootstrap aggregating, boosting, and Bayesian Model Averaging—provides better overall predictive performance and exhibits less bias than any of the ensemble's individual constituents because the aggregate is derived from a weighted combination of all of the ensemble members.
While aggregates provide more accurate estimates for prediction and forecasting, these aggregates do not help modelers refine and improve the individual models that initially formed the ensemble. There is no known method that utilizes the benefits of aggregation techniques to provide feedback to modelers so that they can better understand the events they are trying to model, characterize, or classify.